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COMPUTER  SYSTEM  COMMAND 


I .  Background 

The  U.S.  Army  Institute  for  Research  in  Management  Information  and 
Computer  Science  (AIRMICS)  entered  into  an  agreement  with  the  Georgia 
Tech  Research  Institute  to  fund  a  three  month  study  titled,  "Modification 
to  Existing  U.S.  Army  Software  Micro-Resource  Estimation  Procedure".  The 
study  was  conducted  by  Dr.  Gerald  J.  Thuesen  of  the  School  of  Industrial 
and  Systems  Engineering. 

The  setting  for  the  study  was  the  Computer  Systems  Command  Support 
Group  at  Fort  Lee,  Virginia. ^(Support  Group  Lee)  This  conmand^is  within 
the  U.S.  Army  Computer  Systems  Command  which  is  responsible  for  the 
design,  development  and  maintenance  of  management  Informations  systems 
for  over  200  military  computer  installations  throughout  the  world. 

Although  organizationally  separated, Support  Group  Lee  is  responsible 
for  all  maintenance  of  computer  programs  utilized  by  the  Army  Logistics 
Command  also  located  at  Fort  Lee.  The  Army  Logistics  command  refered  to 
as  a  proponent  agency  prepares  and  submitts  requests  for  program  modifi¬ 
cations  to  Support  Group  Lee.  These  requests  are  then  routed  to  the  divi- 
.sion  within  Support  Croup  Lee  that  is  responsible  for  the  programs  affected. 

To  manage  the  change  request  from  its  receipt  to  final  disposition  it 
is  necessary  to  estimate  the  resources  that  will  be  required  to  make  the 
modifications  requested.  These  estimates  are  usually  prepared  by  the  systems 
analyst  or  programmer  wlio  is  familiar  with  the  affected  programs.  Then  the 
individual  requests  are  grouped  into  System  Change  Packages  (SCP's)  with 
the  approval  of  the  proponent  agencies.  These  SCP's  are  then  the  basis  for 
commitments  to  the  proponent  agency  regarding  the  time  to  completion.  Also, 
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the  SCP's  form  the  basis  for  controlling  and  managing  within  Support  Group 
Lee  the  resources  necessary  to  perform  the  modifications. 

The  present  method  of  resource  estimation  for  the  individual  change 
request  uses  a  micro-estimating  technique  that  is  embodied  in  the  USACSC 
Form  50.  (See  Ap^iendix  A)  The  procedure  utilized  on  this  form  requires 
numerous  decisions  regarding  program  complexity,  the  quality  and  avail¬ 
ability  of  resources,  knowledge  required  and  the  extent  of  the  change. 

In  addition,  predetermined  factors  are  used  to  account  for  indirect  time 
related  to  the  task.  The  result  of  selecting  various  values  for  each 
element  on  the  form  and  combining  these  elements  as  described  yields  an 
estimate  in  man-days  of  the  time  to  do  the  job.  This  estimate  provides 
the  elapsed  time  to  accomplish  the  job  rather  than  the  direct  time  that 
would  be  expended  executing  the  actual  work. 

Raven  Systems  and  Research,  Inc.  was  assigned  the  task  to  study  the 
current  estimating  procedure  based  on  the  Form  50.  Their  conclusions  and 
recommendations  are  included  in  their  August  1979  report  to  AIRMICS  (2  ). 
Essentially,  the  report  discusses  how  the  users  view  Form  50  and  its 
resulting  estimates,  the  accuracy  of  those  estimates,  and  the  difficulties 
of  varifying  those  estimates. 

Since  the  interval  of  the  Raven  Systems  study  overlapped  the  initia¬ 
tion  of  this  study.  Dr.  Thuesen  worked  directly  with  the  participants  in 
the  Raven  System  study.  Although  most  of  the  data  gathering  had  been 
accomplished.  Dr.  Thuesen  and  Dr.  Ronald  Askin  participated  in  the  design 
of  the  modified  Form  50  proposed  in  the  Raven  Systems  report.  Although 
the  basic  approach  was  not  changed,  the  procedures  were  considerably 
simplified  to  at  least  reduce  the  computational  errors  being  Introduced 
by  the  more  complex  Form  50. 
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Tlu-  task  of  this  study  is  to  utilize  the  data  gathered  by  Raven 
Systems  and  to  develop  an  improved  method  of  micro-resource  estimation. 

Tliat  is,  we  would  investigate  whether  an  improved  estimating  methodology 
could  he  developed  so  that  better  resource  planning  and  control  would  be 
possible.  This  new  methodology  would  hopefully  replace  the  present  approach 
represented  by  Form  50  and  expand  the  application  of  the  estimates.  A 
major  difficulty  pointed  out  by  the  Raven  Systems  study  was  the  inability 
to  elicit  the  actual  man-days  to  perform  a  particular  SCR.  Therefore, 
any  approach  utilized  in  this  study  could  not  be  based  on  any  comparisons 
of  actual  versus  estimated  data.  Therefore,  this  study  not  only  proposes 
a  new  method  of  estimating,  but  also  a  procedure  for  validating  the  new 
approacli  that  is  suggested.  This  validation  would  compare  on  a  statistical 
basis  actual  time  with  estimated  time. 

It.  Procedures 

The  procedures  utilized  in  this  study  included  two  phases.  The  first 
phase  is  primarily  concerned  with  understanding  how  Support  Group  Lee 
tunciio-.s  and  familiarization  with  the  literature  that  seemed  applicable 
to  the  problem  of  estimating  resources.  Dr.  Thuesen  and  Dr.  Askin  attended 
meetings  within  AIRMICS  at  Georgia  Tech  for  orientation  and  considerable 
tim<-  was  spent  witii  James  Gantt  of  AIRMICS  in  discussing  how  Support  Group 
I, Cl’  I'perated.  i'lion,  as  Raven  Systems  finished  their  Interviews,  we  held 
apfirox imately  five  meetings  with  their  personnel  to  get  a  more  detailed 
description  of  how  the  micro-estimating  procedure  was  presently  operating. 

At  ttie  same  time.  Dr.  Thue  en  and  Dr.  Askin  were  investigating  the 
literature  with  regard  to  techniques  that  would  be  applicable  to  the 
micro-estimating  problem  faced  by  Support  Group  Lee.  Because  almost  all 
the  workload  at  Support  Group  Lee  consists  of  modification  of  computer 
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programs  rather  than  program  development,  an  attempt  was  made  to  find 
existing  methods  that  would  be  applicable-  One  document  that  describes 
in  detail  the  present  methodologies  for  resource  estimation  is  a  1978 
report  by  H.  M.  Wadsworth  (3  ).  Unfortunately,  none  of  the  methods 
discussed  in  this  report  seemed  applicable  to  the  resource  estimation  of 
computer  program  maintenance. 

The  second  phase  of  this  study  was  focused  on  developing  a  new  approach 
for  micro-estimating  and  determining  the  method  of  validation  that  should 
be  followed.  This  phase  required  two  trips  for  Interviews  with  personnel 
In  Support  Group  Lee.  The  first  trip  was  used  to  determine  whether  the 
modified  Form  50  proposed  by  Raven  Systems  would  be  sufficient.  After  the 
interviews,  it  was  clear  that  an  approach  that  estimated  direct  man-days 
rather  than  elapsed  man-days  would  be  more  useful  for  control  and  management 
of  the  SCP's. 

The  second  trip  was  utilized  to  elicit  suggestions  and  comments  con¬ 
cerning  the  new  approach  that  had  been  developed  since  our  first  contact 
with  Support  Group  Lee.  Both  project  leaders  and  programmers  were  shown 
two  dif‘’crent  forms  based  on  the  same  basic  approach.  The  form  in  Appendix 
B,  which  was  finally  adopted,  requires  considerably  less  detailed  esti¬ 
mating  than  the  form  shown  in  Appendix  C.  The  more  detailed  procedure 
shown  in  Appendix  C  was  generally  rejected  by  those  who  would  have  been 
responsible  for  executing  the  estimating  procedure.  The  concensus  opinion 
was  that  it  is  not  realistic  to  separate  the  various  inputs  into  such 
small  elements.  Thus,  no  improvement  in  the  final  estimate  would  be 
achieved  by  making  the  additional  effort  to  work  with  the  larger  number 
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In  addition,  discussions  concerning  the  best  approach  for  validating 
thi'  new  estimating  procedure  were  had  with  those  who  would  be  directly 
involved  in  any  new  estimating  methodology.  It  was  concluded  that  the 
basic  estimating  unit  should  be  each  system  change  request.  (SCR)  By 
recording  the  estimates  made  for  each  SCR  before  the  work  is  performed 
and  utilizing  the  project  management  system  (PMS)  to  record  the  actual 
effort  expended,  data  for  validation  would  be  available  for  the  first 
t  ime. 

The  data  resulting  from  this  data  collection  activity  would  then 
he  analyzed  to  test  for  significant  statistical  deviations  between  the 
actual  and  estimated  values.  The  statistical  tests  to  be  applied  have 
been  determined  as  part  of  this  study  and  they  will  be  discussed  in  detail. 

Tlic  final  task  in  the  second  phase  of  this  investigation  will  be  the 
preparation  of  the  final  report. 

III.  Analysis 

Before  the  initial  trip  to  interview  personnel  for  this  study, 

Dr.  Thuesen  and  Dr.  Askin  held  four  lengthy  sessions  with  Robert  Barrier 
and  Gaye  Stewart  of  Raven  Systems.  The  purpose  of  these  meetings  were  to 
gather  information  pertinent  to  the  study  to  be  undertaken.  Since  it  was 
known  tliat  the  final  report  to  be  prepared  by  Raven  Systems  would  not  be 
compli-ted  by  the  time  our  study  was  initiated,  we  worked  Informally  to 
undiTstand  the  status  of  the  micro-estimation  procedure  that  was  currently 
In  operation  at  Support  Group  Lee. 

A  number  of  Important  Issues  were  identified  and  it  is  these  Issues 
that  became  the  basis  for  our  proposed  solution  to  the  micro-estimating 
process.  It  became  evident  that  any  micro-estimating  procedure  must 
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provide  information  that  was  meaningful  for  aggregating  the  change  requests 
into  the  System  Change  Packages.  (SCP)  This  process  described  as  a  ".scrub" 
session  allowed  the  proponent  agency  and  the  personnel  at  Support  Croup  Lee 
to  prioritize  the  change  requests  which  were  to  be  Included  in  the  new  SCP'-. 
At  this  time,  the  estimates  of  man-days  determined  by  the  Form  SO's 
are  utilized  to  provide  some  Indication  of  the  resources  that  would  be 
required  by  Support  Group  Lee  to  accomplish  a  particular  job.  The  estimates 
that  were  used  in  this  process  were  sometimes  modifications  of  the  Net 
Development  Time  as  shown  on  the  Form  50.  These  modifications  were  based 
on  a  reduction  of  the  Net  Development  Time  obtained  by  multiplying  by  fac¬ 
tors  of  0.4  to  0.5.  These  factors  based  on  the  experience  of  team  leaders  at 
Support  Group  Lee,  gave  us  a  strong  indication  that  those  team  leaders  who, 
according  to  the  Raven  Systems  people,  were  more  sophisticated  in  their 
management  techniques  realized  that  the  Form  50  estimate  was  not  accurate. 

Because  we  had  no  data  on  which  to  verify  these  opinions,  we  estab¬ 
lished  the  position  that  team  leader  experience  will  approximate  the  actual 
outcome  with  a  good  degree  of  confidence.  This  conclusion  is  based  on  the 
consulting  experience  of  Dr.  Thuesen,  who  has  observed  numerous  instances 
where  approximate  methods  based  on  experience  usually  provides  results  close 
to  what  might  have  been  achieved  with  more  systematic  analysis,  (i.e.  Common 
sense  tempered  with  experience  leads  to  near  optimal  solutions) 

Based  on  these  observations,  it  was  concluded  that  the  Form  50,  which 
provided  total  elapse  time,  was  not  the  appropriate  information  for  estab¬ 
lishing  change  request  priorities  in  the  "scrub"  session.  It  was  the  direct 
hours  associated  with  the  change  requests  that  would  be  more  useful  in  this 
planning  activity. 
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Once  an  SCI’  i.s  assembled,  the  other  important  function  tliat  is  depen¬ 
dent  on  good  micro-estimating  is  project  control  or  project  management. 

The  SCP  becomes  the  basic  unit  for  determining  milestones  and  final  deli¬ 
very  dates.  According  to  the  Raven  Systems  people,  more  preestablished 
milestones  for  SCR's  are  missed  than  are  achieved.  Some  of  these  mile¬ 
stones  have  been  mi.ssed  by  more  than  200%.  Tt  was  our  impression  based  on  con¬ 
versations  with  Raven  Systems  personnel  and  Support  Group  Lee  personnel, 
that  this  inability  to  meet  stated  deadlines  was  the  most  serious  problem 
faced . 

Accepting  this  premise  then  caused  us  to  examine  how  Improved  control 
of  the  SCP's  could  be  accomplished.  In  our  discussions  with  Raven  Systems 
and  Support  Group  Lee,  it  was  recognized  that  the  problem  of  project  control 
had  many  facets;  one  of  which  was  the  lack  of  accurate  time  estimates  for  the 
SCl’'s.  However,  the  most  serious  problem  seemed  to  be  related  to  the  wide  vari- 
t  t\'  or  l.iei  of  methods  used  by  tlie  various  project  leaders  for  project  control 
and  resourci’  planning.  Although  the  Projec  Management  System  (PMS)  is 
availalile  and  encouragement  is  given  by  upper  management  for  its  use,  only 
a  fi'w  individuals  were,  in  fact,  using  this  system.  Since  PMS  is  designed 
witii  great  user  flexibility,  each  user  has  tailored  his  own  individual 
report  ing  ■  cheme  and  what  use  is  occuring  is  non-standardized. 

Bi’cau.se  PMS  is  a  powerful  management  tool  and  because  an  even  more 
useful  Decision  Support  System  is  currently  under  development  by  AIRMICS, 
we  concluded  that  any  micro-resource  estimating  technique  that  is  developed 
should  provide  the  Information  in  a  form  that  increases  the  efficiency  of 
tliese  nuinagement  tools.  The  data  form  which  allows  greatest  utilization 
of  PMS  is  to  input  direct  man-days  for  task  completion.  Direct  man-days 
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represent  the  estimate  of  actual  hours  to  be  spent  on  Review  and  Analysis, 
Design,  Communication,  Coding,  Level  I  testing  and  Documentation.  Addi¬ 
tional  data  regarding  the  future  availability  of  resources  due  to  planned 
vacations,  scheduled  leaves,  and  other  anticipated  manpower-reductions  are 
then  input  directly  into  PMS .  This  approach  allows  eacii  unit  to  plan 
more  reallstlcly  according  to  their  specific  resource  availabilities. 

The  approach  on  which  the  Form  50  is  based  provides  total  elapsed 
time  that  is  determined  by  using  constant  factors  for  the  estimation  of 
resources  required  by  the  non-direct  activities.  As  a  result,  each  unit 
has  less  ability  to  recognize  their  own  peculiarities  regarding  their 
non-direct  resource  requirements.  In  addition,  there  was  no  indication 
that  the  Form  50  methodology  would  provide  resonable  estimates  of  the 
direct  time.  Therefore,  it  was  concluded  that  a  new  method  needed  to  be 
developed  that  would  focus  on  micro-resources  estimation  of  direct  man¬ 
power  requirements. 

IV.  Results 

Working  from  the  basic  premise  that  the  appropriate  information  to  be 
provided  by  any  micro-estimating  procedure,  should  be  in  terms  of  direct 
effort  expended,  an  entirely  new  estimating  procedure  was  developed.  This 
procedure  requires  that  estimates  be  made  in  such  a  manner  that  each  indi¬ 
vidual  contribution  to  the  direct  work  effort  performed  be  included. 

Because  the  estimates  are  to  be  in  direct  man-days,  the  estimates  for  each 
of  the  component  activities  required  to  complete  the  SCR  will  be  additive. 
Thus,  a  rather  simple  arithmetic  procedure  has  evolved  which  will  be  useful 
in  the  estimation  of  resource  requirements  for  both  the  SCR's  and  the  SCP's 
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I  ’.iU  iuMl  iii^  I  ho  SCK 

IKo.iuso  t  lio  most  Ijasic  work  task  Is  tiio  program  arfoctcd  In  each 
svsti'in  iliango  rotpiest  (SCR),  it  was  docldod  that  this  basic  unit  should 
hi'  I  lu'  basis  lor  mi  c  ro- resource  i-stimating.  Witli  the  possibility  of  one 
or  inori'  individuals  being  directly  involved  in  tlie  work  activity,  tlie 
decision  was  made  tliat  a  direct  man-day  estimate,  for  each  individual  be 
recorded.  Tiius ,  if  the  nature  of  the  work  indicates  that  a  systems 
analvst  and  two  programmers  will  be  involved,  three  separate  estimates 
are  to  tie  made.  f.aeli  estimate  sliould  be  produced  by  each  of  the  individuals 
in  light  of  thi‘  tasks  tliat  eacli  anticipates  being  retpiired  to  perform.  If 
this  level  of  knowledge  about  wlio  is  to  perform  what  specific  tasks  is  not 
.iva  i  lab  1 1' ,  then  a  sin.v.le  i.'stimate  will  be  made.  The  general  rule  is  that 
.IS  manv  ii.t  imat  es  will  he  made  for  each  program  within  a  SCR  as  there  are 
individuals  who  are  actually  required  to  make  inputs  regarding  their  esti¬ 
mated  functional  involvement  in  performing  work  on  that  particular  pro¬ 
gram. 

The  form  prtisented  in  Appendix  B  will  be  the  mechanism  for  recording 
the  I'st  iraati's  of  the  individuals  involved.  Thus,  not  only  must  the  system, 

:>CK  iinml'i'i  .  .iiid  progr.iiii  .il  ifeted  ln>  identified,  hut  e.ich  contributor  to 
tlie  overall  I’st  i  iiiati-  must  also  be  known. 

Alter  con.s  iiler.ib  le  study  and  review  by  personnel  .U  Support  Croup  hoe, 
six  important  categories  of  work  effort  were  identified.  These  six  categories 
iMicompass  tin.'  significantly  different  types  of  work  activities  that  are 
iisii.ills  .issue  i. It  ed  with  .a  sy.sli'ra.s  c.liange  request  (SCR).  Because  it  was 
recognized  that  the  individuals  that  Impact  on  a  SCR  may  be  performing 
different  function.s,  it  is  Important  tliat  each  category  be  interpreted  with 
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respect  to  each  individuals  contribution.  That  is,  a  systems  analyst  may 
view  the  work  effort  required  for  the  Review  and  Analysis  category  quite 
differently  than  a  programmer  would  view  the  same  category,  i he  systems 
analyst  may  hi'  reviewing  the  system  of  prog.r.ims  Inliiy  .illi'iii.l  wh  i  I  i  I  In 
programmer  would  he  spi'iullng  tils  review  el  loi  l  In  iiiulet  si  anil  Ing,  hew  i  he 
specific  program  for  wtiich  he  is  res))onslble  would  hi'  changed. 

B.  Estimating  Categories 

The  six  estimating  categories  on  which  the  proposed  micro-resource 
estimating  procedure  is  based  are: 

Review  and  Analysis 
Design 

Communication 

Coding 

Level  I  Testing 
Documentation 

Each  individual  involved  in  the  estimating  procedure  should  make  estimates 
only  for  those  categories  in  which  a  direct  contribution  to  tlie  completion 
of  the  task  will  be  made.  For  each  category  estimated  by  an  individual,  a 
single  value  representing  the  anticipated  direct  man-days  require  will  be 
recorded.  If  there  are  categories  for  which  an  individual  has  no  involve¬ 
ment,  his  estimate  should  be  left  blank  for  that  category.  It  is  antici¬ 
pated  that  a  Systems  analyst  will  not  normally  contribute  In  the  Coding 
category,  so  no  entry  sliould  he  made. 

It  should  be  pointed  out  that  activities  such  as  Level  IT  Testing, 
Level  III  Testing,  Environmental  Test  and  Field  Validation  are  considering 
outside  the  proposed  estimating  procedure.  Because  these  activities  are 
generally  handled  differently  and  they  are  much  more  dependent  on  avail¬ 
ability  of  computer  time,  it  is  believed  that  a  different  approach  must  be 
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lollDwod  ill  oritor  to  dovolop  accurate  estimates  for  tliese  activities.  How¬ 
ever,  it  Is  important  tliat  accurate  estimates  of  those  activities  need  to 
be  available  if  sound  planning  and  control  is  to  be  achieved  at  Support 
lirotip  I.ee. 

I.  Review  and  Analysis 

In  general,  this  activity  represents  all  the  preparatory  work  that  is 
necessary  before  any  decision  regarding  implementation  of  the  change  request 
can  be  made.  Fur  tiie  System  Analyst,  this  would  include  tlie  effort  spent 
l  O  CM  d  i  nal  lug  and  comniun i cat ing  with  the  proponent  agency  to  Insure  there  is 
an  accurate  unders t and ing  of  the  request.  In  addition,  time  spent  studying 
tlie  relation  of  tlie  change  request  to  the  system  of  programs  affected  would 
be  included  in  this  category. 

tin  the  other  hand,  this  category  represents  for  the  programmer  the 
effort  to  lie  expended  in  understanding  the  program  files  and  logic  to  be 
affected  bv  tlie  change  request.  This  estimate  should  include  studying 
program  listings,  locating  documentation,  recording  the  nature  of  the 
cliange  rec|uest  ,  and  understanding  the  scope  of  the  problem. 

J.  Dcuiigii 

iiii;.  c-at  cg.oiy  is  concerned  with  the  effort  tiiat  is  to  he  expended  In 
translating  tlie  c  lian);e  rec|uest  Into  sjiecific  remedies  that  will  insure 
successtul  completion  of  the  task.  Primarily,  this  category  will  recog¬ 
nise  the-  time  involved  in  synthesis,  and  the  development  of  specific  change 
specifications.  It  is  anticipated  that  both  system  analysts  and  programmers 
will  be  involved  in  this  type  of  activity.  The  systems  analyst  will  be 
involved  with  design  ri’garding  how  the  change  request  affects  the  system 
of  programs  wliili'  tlu'  programmer  will  be  concerned  with  how  the  change 
recpiest  .iftects  his  particular  program. 


3 .  Conunun ^ at  ion 

Thf  effort  that  arises  from  the  interaction  between  tlie  system  an.ilvst  an.l 
prograimner  to  assure  that  the  changes  to  each  of  tin'  sv.sti-m  parts  will  lea. I  !<> 
the  overall  objective  of  t  lu'  change  rei|uest  will  be  recordial  as  eommuii  i  t  at  ion . 
This  activities  will  include  both  written  and  verbal  communication.  Wlien 
this  type  of  activity  occurs,  tlie  systems  analvst  should  i-stimate  h  U;  iiivi'lv.  - 
ment  (he  may  be  involved  with  a  number  of  programmers  for  a  jiart  iciilar  S('.K) 
and  the  progra(nmer  will  estimate  his  effort  in  this  activitv. 

U .  t^od  ijii> 

Under  this  eati'gorv  is  recorded  all  the  effort  exiiended  by  the  pro¬ 
grammer  to  translate  tlie  change  specifications  into  coded  logic.  Given 
the  change  specifications,  what  coding,  file  manipulation,  and  other  pro¬ 
gramming  activities  are  necessary  to  produce  a  source  deck  that  is  ready 
for  testing. 

3.  Level  I  Testing 

This  category  reflects  all  tlte  effort  that  is  requirisl  to  test  the 
program  that  has  been  modified  to  satisfy  a  change  reqiu'st  f.o  that  the  pro¬ 
gram  will  he  ready  lor  Level  II  Testing.  Tlie  effort  recorded  should  refli'cl 
activities  which  include  compilation,  development  of  test  data,  analysis  of 
test  results  and  corrections.  It  Is  important  for  this  category  that  only 
direct  effort  expended  by  considered  so  that  delays  due  to  machine  avail¬ 
ability  are  not  part  of  the  estimate.  Machine  dependent  delays  should  be 
considered  as  indirect  effort  that  is  required  to  satisfy  the  change  request. 

6 .  Do  cum ontat  i^n 

Any  documentation  effort  associated  with  a  complcti'd  change  reijiiest  will  he 
recorded  here.  Documentation  includes  any  changes  to  operating  or  system 
manuals  and  any  documentation  as.sociated  with  changes  in  program  logic. 

Again,  only  the  direct  effort  expended  by  the  system  analysis  or  programmer 
slcaild  be  considered  here. 

Z'  ■  - - 

/ 
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III  iitiditioii  to  iiu‘  i  111  orinci  I  ion  tli.it  apponrs  on  tlu’  front  of  the 
Direct  Man-Day  Hstimating  Form,  our  d  Lscuss ions  with  Support  Group  Lee 
|)ersonni'l  revealed  tliat  some  written  description  of  the  nature  of  the 
ehannes  that  were  llu‘  basis  for  the  estimate  would  be  extremely  useful. 
Because  the  original  estimate  for  the  SCR  is  made  months  in  advance  of 
the  ini' lus ion  ot  that  SCK  in  a  SCP,  it  was  usual  that  the  person  making 
the  esl imate  was  not  the  same  person  actually  implementing  the  change 
reipiest .  All  personnel  we  questioned  insisted  that  if  the  person  making 
the  orij’.inal  estimaU’  would  record  In  writing  a  description  of  the  program 
changes  necessary  to  satisfy  the  change  request,  much  greater  under¬ 
standing  of  the  estimates  would  be  possible.  Also,  the  written  ri’cord 
w('u!d  provide  a  eontinuity  so  llial  tlie  person  who  must  aetuallv 
i  mi' leiiK'iit  the  change  request  will  not  have  to  begin  his  analysis  from 
scratcii. 

This  narrative  description  of  anticipated  changes  includes  a  place 
to  list  tlie  I'iles  that  would  he  affected  by  the  SCR.  Again,  this  informa- 
I  ion  is  intended  to  help  validate  the  estimate  and  to  provide  continuity 
to  till.'  istimat  ing,  proci’ss. 

C.  Kst  iin  It  i  iig  the  Systems  Clianyye  Package  (SCP) 

The  individual  (.'stimates  on  tlie  SCR's  are  intended  to  bo  the  basis 
for  determining  delivery  times  for  a  SCP.  It  is  anticipated  that  following 
till'  "scrub"  session  milestone  estimates  would  be  developed  by  combining 
the  direct  man-day  estimates  for  the  SCR's  contained  in  a  SCP  with  the 
indirect  man-davs  estimated  for  all  personnel  activities  not  directly 
.1 1  I  r  i  liii  t  ah  1  e  to  the  eliange  request.  These  estimates  arc  then  input  to  the 
I’rojct  t  Manaj'.eiiieiii  System  (PMS)  so  that  the  total  resources  can  be  scheduled 
Hill  I  e.isoiiahl  e  niilesiones  for  completion  of  system  change  (lackages  can  be 
( i xed .  iiils  process  should  be  relatively  straight  forward  as  the  direct 


man-day  estimates  resulting  for  the  new  form  will  be  additive  with  thc" 
Indirect  man-day  estimates. 

D .  Testing  the  Validity  of  Estimating  Technique 

The  validation  oi  tlu'  m  ic  ro- resource  I'sl  ima  l  i  ii)’,  i  icliii  i  cuie  piopofed 
in  tills  stinly  sliould  li.ivi'  two  distinct  t>ein'fils.  I'li’st,  t  Iw  esl  iitiii.  . 

:iuule  jirlor  to  the  Inili.itioii  of  tile  work  c.iii  he  stat  ist  iiallv  couie.it.a 
with  tile  actual  i  t  tort  ri'corded  lor  that  work.  For  each  ot  these  cate 
gories  a  statistical  comparison  will  be  made  to  see  how  well  each  oi 
these  type  of  estimates  can  be  made.  Also,  the  total  man-days  estimated 
will  be  compared  with  the  actual  total  effort  expended  so  that  the  pro¬ 
cess  of  summing  the  categories  to  find  aggregate  estimates  can  be  validated 
The  second  important  effect  of  recording  estimated  and  actual  effort 
expended  is  the  increased  awareness  that  will  be  engendered  in  the  Support 
hroup  bee  personnel.  This  awareness  should  encourage  more  serious  consider 
t  ion  of  the  individual's  role  In  making  rej  i.iblc  istlm.ilcs  when  using,  i  he 
proposi-d  proci'dure.  Many  of  tlie  persons  interviewed  fi'lL  that  this  would 
be  one  of  tlie  most  positive  outcomes  of  this  study. 

1 .  Test  Procedures 

Tj^  shall  represent  the  time  (or  the  form  initiated  at  that  time)  at 
which  an  SCR  is  first  obtained  from  the  proponent  agency. 

siiall  represent  the  time  (or  the  form  recorded  at  that  time)  immcdl 
ately  before  the  scrub  session.  (It  is  assumed  that  at  T^  the  person  or 
persons  who  will  be  working  on  tlie  SCR  can  be  Identified) 

Tj  shall  represent  the  time  (or  data  recorded)  just  after  completion 
of  the  work  on  the  SCR.  (Actually,  as  categories  of  work  are  completed, 
the  (>ffort  will  be  recorded  and  input  to  PMS) 
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2.  Data  C’olh'ction 

I'wn  learns,  SAAS  DVL  [  headed  by  Mr.  R.  A.  Del.ong  and  A(-)/ACS  Team 
by  Mr.  I.  T.  I'litton,  have  been  .selected  by  management  at  Support  Group  Lee 
to  providi-  tlie  test  environment  for  the  proposed  micro-estimating  procedure. 
Both  groups  will  be  utilized  to  provide  estimated  and  actual  data  and  it 
is  intended  that  this  data  be  accumulated  for  approximately  one  year.  This 
^.llulv  piTiod  will  allow  for  the  inclusion  of  3  to  4  completed  SCP's  for 
I'.ieh  team.  Based  on  the  normal  size  of  an  SCP,  it  is  anticipated  that  150 
to  25(1  separate  estimates  will  be  required  providing  a  solid  data  base. 

I  i.i  I  a  Go  1_1  i.'e  L_ui_n 

.1.  At  timi'  Tj^  any  involved  systems  analyst  and  programmer  will 
fill  out  separate  estimating  forms  for  each  program  affected 
by  the  SCR.  These  will  then  be  filed  together  as  a  combined 
estimate. 

b.  At  time  T2  the  systems  analyst  and  any  programmers  who  are 
actually  responsible  for  implementing  the  SCR  will  fill  out 
si-parate  estimates  on  the  form.  There  will  be  a  form  required 
lor  each  program  affected  by  the  SCR.  These  estimates  will 

be  filed  together  and  the  sum  of  the  individual  estimates 
will  represent  the  total  direct  time  estimated  for  the  SCR. 

c.  As  soon  as  the  actual  work  commences  on  the  SCR,  the  actual 
direct  time  expended  in  each  o'  the  six  categories  will  be 
recorded  daily.  These  daily  records  will  then  be  the  basis 
for  weekly  reporting  of  actual  effort  expended.  By  utilizing 
the  Project  M;in.igement  System  (PMS)  the  weekly  totals  of 
actual  direct  man-days  allocated  to  each  of  the  six  categories 
will  be  available  for  analysis.  (It  is  Important  that  the  same 
lonu.it  and  ti’portinjt  categorie.s  bo  u.sed  in  PMS  by  all  participants 
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in  I  lie  stnilv)  At  I  1  ini'  I' ^  i'oin|).i  r  I  sum;  nl  .nlii.il  illnrl  vnsm. 
I'stiniatod  oltort  will  he  made. 

3.  Test  of  Direct -Man  Da^  Es^ijnatj nj>^  J^jrm 

By  having  estimates  at  and  and  actual  values  at  1^,  there  are 
two  primary  premises  to  test.  Comparing  actual  effort  expended  to  the 
estimates  made  at  T2  we  have  a  test  of  estimating  proficiency  in  the  most 
favorable  of  circumstances.  That  is,  the  individuals  who  are  actually 
assigned  to  do  the  work  will  be  making  the  estimate  immediately  prior  to 
starting  itie  job,  hence  the  same  lndividu.il  will  be  m.iking,  the  esi  im.iti*  and 
iloing,  the  work.  It  Is  assumed  th.at  ,111  indlvidnal  c.in  best  estimale  the 
I'ffort  he  must  expend  for  satisfactory  completion  of  the  job.  We  believe 
that  the  individual  knows  his  own  capabilities  better  than  someone  else. 

The  second  premise  to  test  is  whether  estimates  made  at  Tj^  are  reli¬ 
able  estimates  of  the  actual  effort  that  is  expended.  In  this  case,  those 
who  are  making  the  estimates  are  not  necessarily  those  who  will  be  doing 
the  actual  work.  Also,  these  initial  estimates  usually  occur  two  to  six 
months  before  the  work  actually  begins  and,  therefore,  there  may  not  he  a 
clear  definition  of  the  actual  tasks  that  must  be  accomplished. 

V.  Statistical  Tests 
A.  Groi^  1 

Test  the  liypothesls  that  the  mean  of  each  category  estimated  at  I',,  is 
equal  to  the  mean  of  the  actual  values  at  T.^.  is  the  number  of  SCK's 

completed  by  Group  1.  The  statistical  test  use  is  a  paired  t  test  (1,  p2A2-<’s6) 
Let  x^j  be  the  estimated  direct  man-days  for  SCR  i  and  cateogry  j 
(1  =  l,2,...,n^)(j  =  1,2,. ...6) 


Group  1 


Group  2 


IS 


2.  Calculate  S 


n  1+112 


Calculate  ‘q  “  ’’j  ■*■  ''•i 

■/“72  \ 


i 

\  "i+"2  / 


4.  Compari'  In  a  L  d  i  st  r  I  hut  eel  random  variable  with  iij  t  ii  ,  -  1 

degrees  of  freedom.  If  IIqI  too  large,  reject  the  liyiuit lies  1  s 
that  the  mean  of  the  total  estimated  effort  for  both  groups 
is  equal  to  the  mean  of  the  total  actual  effort  expended  by 
the  two  groups. 

D.  Comparison 

To  test  the  effectiveness  of  providing  reliable  estimates  by  esti¬ 
mating  at  Tp  compare  the  data  with  the  data.  All  the  tests  described 
in  A,  B,  and  C  would  be  repeated  for  this  different  data  set. 

E .  Scatter  Diagram 

An  additional  technique  for  investigating  visually  the  relationship 
between  the  actual  values  and  the  estimated  value  can  be  accomplished 
through  the  use  of  a  scatter  diagram.  That  is,  plot  the  estimated  value 
for  each  data  point  against  the  actual  value  for  that  point.  An  example 
of  a  scatter  diagram  is  shown  below: 


Estimated 


Actual 
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TiiiH  technique  i.s  useful  in  providing  insight  regarding  the  bias  asso¬ 
ciate  with  data  where  it  has  been  shown  statistically  that  the  means  are 
not  equal  with  any  statistical  confidence. 

'  .  .  , ,  '  ' 

VI.  Conclusions  .  — ■  ^  ' 

^  V  V  4  y  ^  ^ 

After  having  studied  the  data  gathered  by  Raven  Systems  regarding 

till'  use  of  the  existing  Form  50, and  having  visited  Support  Group  Lee 

■  .1  ■ 

<ind  observed  how  they  operate,  the  folio- 'ing  recommendations  are-'suggested 
Tlu'se  recommendations^ are  focused  strictly  on  the  task  of  micro-resource 
est iraat ion  * 

1^,  Form  50,  the  existing  mechanism  for  estimating  resources 
expended,  should  be  replaced  by  an  estimating  methodology 
based  on  the  direct  man-days  required  to  accomplish  the 
task!  The  new  format  for  these  estimates  is  presented 
in  Appendix  B. 

2.  Estimates  should  be  made  by  the  person  or  persons  who 
will  actually  make  the  change  immediately  prior  to  the 
initiation  of  the  work.  If  estimates  must  be  made  some 
time  in  advance,  new  estimates  should  be  made  just  prior 
to  undertaking  the  workj 

3.  The  Project  Management  System  should  be^-utllized  to  track 
the  direct  and  indirect  effort  associated  with  each  System 
Change  Request  fSCR).  This  record  keeping  will  ensure  the 
availability  of  data  so  that  improvements  in  the  micro- 
estimating  process  will  result. 
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4.  Statistical  verification  of  the  new  methodology  will  be 

accomplished  by  the  study  over  the  period  of  approximately 
one  year  of  actual  vs.  estimated  man-days  required.  Two 
teams  of  Support  Group  Lee,  SAAS  LVL  I  and  A(-)/ACS,  have 
been  selected  to  provide  as  the  test  environment  for  the 
verification  of  the  proposed  procedures.  This  study  will 
provide  the  first  reliable  data  that  should  be  the  basis 
of  any  well  constituted  estimating  procedure. 
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APPENDIX  A 


IMPACT  ESTIMATING 

For  m>  of  «h(i  form,  tmm  USACSC  SGL  Matno  18-t  (Chap  6(;  lfia'pro(<onant  aparrcy  la  USACSC  Spt  Op,  Ff  Laa  lOAO) 


SYSILM  _ 

SCR/PHOGRAM 


ESTIMATOR'S  NAMEIS) 
DATE 


This  form  is  to  be  used  lor  impacting  man-days  effort  required  for  implementation  of  the  above  SCR/program.  Standard 
(actors  are  shown  below.  This  form  is  to  be  attached  to  USACSC  Form  6. 


SECTION  1 

1.  INPUT  FILE  FORMATS  AFFECTED  BY  THIS  SCR 

a.  Number  of  card  files  _ 

b.  Number  of  tape  files  _ 

c.  Number  of  disk  files  _ 

Number  X  Factor 

X  1  - 

XI  - 
XI  - 

_  TOTAL 

2.  OUTPUT  FILE  FORMATS  AFFECTED  BY  THIS  SCR 

a.  Number  of  card  files  _ 

XI  - 

b.  Number  of  tape  files 

XI  - 

c.  Number  of  disk  files  .  _ 

_  X  1  - 

d.  Number  of  reoort  formats 

X  1  - 

TOTAL 

3.  PROGRAM  FUNCTIONS*  NOTE:  This  table  reflects  the  number  of  programs  which  inclu4e  tunctions  ^(ected  by  the 
SCR,  (e.g..  An*3CftTnay  affect  an  edit-validation  fursction  irt  each  of  3  programs.  Two  are  simple,  one  is  complex.  Enter: 
a.  Edit  validation  4  X  2 ' 8  8X1-8  12X0-OJ 


a.  Edit-validation 

b.  Sort/merge  process 

c.  Internal  data  manipulation 
'  d.  File  search 

.e.  Tfble  look-up  (internal  or  external) 

-f.  Calculatioru 

-g.  Utilities  or  subroutines 

‘h.  Job  Control  languages 

-  .  Subtotals 

r  Total  of  Program  Furtctions 


SIMPLE 
Factrsr  X  Pgms 
4X  - 


COMPLEX 
Factor  X  Pgms 

8X - 

3X - 

3X - 

3X _ 

5X _ 

5X _ 

3X _ 

2X  _ 


VERY  COMPLEX 
Factor  X  Pgms 
12  X  - -  - 


USACSC-FT  LEE  FUHM  bO 

1  MAR  77 


7.  PROGRAM  TURN  AROUND  TIME  (Average)  ~1 

FACTOR 

a.  Effective  lAP  Usage 

0.6 

f  .  More  than  once  per  day 

0.8 

c.  Cnee  per  day 

1.0 

d.  Less  than  once  per  day. 

1 

1.2 

9.  DOCUMENTATION  CHANGES  REQUIRED  BYTHIS  SCR  1 
/ 

Number  of  pages  to  be  changed/added 

8.  SYSTEM  EACTOH 

FACTOR 

•-  Developmental  "  2 

b.  Major  change  3 

c.  Major  modification  4 

d.  Minor  modification  5 

e.  Maintenance  6 

I.  Minor  technical  change  7 

g.  JCL  change  only  8 


1.  (1) 


SECTION  II 

NET  DEVELOPMENT  TIME 
(3)  (3a) 


Input  Total  Output  Total  Program 

Function  Total 


Sub-Total 


NOTE:  If  |3a)  is  zero,  enter  one. 


Total  from  Resources 

(3a)  Average 


(6)  (6)  (7)  (7,) 

Job  Knowledge  Job  Knowledge  Program  Turrv  Sub-total 

Required  Available  around  Factor 

X  XX- 


Total  from 
(7e) 


(8)  (9)  (10)  (11)  (12)  (13) 

System  Development  Other  System  Non-Project  Lost  Time  Net  Development 

Factor  Time  Factor  Factor  Factor  Time 


X  1.8 


X  I1J25 
2.43 


♦  0.1) 


man-days 


Total  of  Column  13  is  entered  onto  Line  #1  of  the  SCR  Estimate  Summary  and  will  be  defined  as  Net  Development  Time 
an  the  SCR  Estimate  Summary. 


[1.  Net  Development  Time  - 


SCR  ESTIMATE  SUMMARY 
man  days 


■.  Review  and  analysis  • 

Ib.  Design 

ic.  Programing  (including  Level  I  testing) 
'8L  Testing  (including  Level  II  &  III  testing) 
Documentation  (enter  zero  for  none) 


■NOT  X  0.15 
■  NOT  X  0.20 
■NOT  X  0.35 
■NOT  X  0.25 
■NOT  X  0.05 


12.  Total  project  man-days  (sum  of  la-e  above) 


man-days  X  8  ■ 


manhours 


I*  Enter  these  figures  in  the  appropriate  blocks  of  the  Impact  Analysis  section  of  USACSC  Form  6,  (System  Change 
fequest) 


I 


SYSTEM 
SCR  NUMBER 


APPENDIX  B 

DIRECT  MAN  HOUR  ESTIMATING 
DATE  _ _ _ 

_  ESTIMATOR’S  NAME  _ 

PROGRAM 


B-1 


1.  Review  and  Analysis  _  Man-hrs. 

(Includes  research,  clarifying  change  request, 
interacting  with  proponent  agency) 

(i.e.  What  is  to  be  done) 


2.  Design  _  Man-hrs. 

(Problem  definition  at  program  level,  synthesis, 
develop  change  specification) 

(i.e.  How  it  is  done) 

3.  Communication  _  Man-hrs. 

(Interaction  between  programmer  and  systems 
analyst ) 


4.  Coding  Man-hrs. 

(All  coding  activities  to  implement  change 
specifications) 

i 

5.  Level  I  Testing  Man-hrs. 

(Compilation,  development  of  test  data, 
analysis  of  test  results  and  corrections) 

Estimated  Number  of  Runs  _ _ 

6.  Documentation  Man-hrs.  I 

(Any  documentation  required  for  completed  \ 

change) 


TOTAL 


Man-hrs . 


NARRATIVE 


1.  Programs  Affected 


APPENDIX  C 


C-1 


DIRECT  MAN  HOUR  ESTIMATING 
DATE 

•''SUM  ESTIMATOR'S  NAME 

si  K  NDMlil'K  _  PROGRAM 


Ri-vicw  anil  Analysis 

Rnsearc  li 

Man-hrs . 

1  tiLiTac  L  i  nj;  with  proponent  agency 

Man-hrs . 

IK'S  i  gn 

I’rohloni  definition 

TOTAL  _ 

Man-hrs . 

Man-hrs 

Svnt  Ill's  i  s 

Man-hrs . 

Devi'lop  spec  i  f  icat  ion 

Man-hrs . 

TOTAL 

Man-hrs 

Coiiumin  i  cat  ion 

TOTAL 

Man-hrs 

4.  Ciuliil^', 


Program  Complexity  j 

Extent  of  Change 

Simple 

Normal 

Complex 

Simple 

Minor 
Mod . 

Major 

Mod. 

Dev. 

a  .  1  /  (•  Korinat  s 

1 

0 

3 

1  ; 

;  2 

3  * 

!  4 

Man-hrs 

h.  Ed  it /Valid. 

1 

2 

3 

1 

2 

3  ' 

1  4 

Man-hrs 

e.  Intern.!  1 

Proi  i’ss  ing 

1 

2 

3 

1  ' 

1 

'  2 

3 

4 

_ _ _ Man-hrs 

d.  Intern. il  D.U.i 
.'l.in  i  pn  1 .1 1  i on 

I 

2 

3 

1  i 

2 

3 

,  4 

Man-lirs 

e  .  1  't  i  le  r  1 

I 

! 

1 

1 

1 

Man-hrs 

TOTAL _ _ 

__  Man-hrs 

Leve 1  I  lest ; ng 

.1 .  Comp  i  l.iL  ion/Ver  i  ca  t  ion 

Man-hrs . 

h.  Development  of  te.st  data 

Man-hrs. 

e.  Valid.it  ion  of  test  results 

Man-hrs . 

TOTAL 

Man-hrs 

Dociiinent.it  ion 

.1 .  U.'.i  r  ll.inu.i  1 

Man-hrs . 

h.  Opel  .It  ion;.  A  .Scheduling  Manual 

Man-hrs. 

e.  I’top.i.iin  logic  dociimi’ntat  ion 

_ _ _ _ _ _ _  Man-hrs. 

TOTAL 

_  Man-hrs 

TOIAI.  .^WN-HRS.  TO  COMPLETE  CHANGE 

Man-hrs 

r-2 


NARRATIVE 


1.  Programs  Affected 


2.  Files  Affected 


3.  Description  of  program  changes  on  which  this  estimate  is  based. 


/ 


E.ND 


DATE 
FILMED 


DTIC 


